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This  paper  relates  a simplicial  pivot  algorithm  o#~th«  -authors '*1:o 
Merrill's  algorithm,  Newton's  method,  and  a ""global  Newton  method"^ pre- 
sented by  Smale,  Some  computational  results  are  given. 


1.  Introduction 

This  paper  is  expository  in  nature.  The  primary  goal  is  to  present 
recently  observed  relations  between  a simplicial  pivot  ailgorithm  of  the 
authors  [10],  Merrill's  algorithm  [19],  Newton's  method,  and  a "global 
Newton  method"  presented  by  Smale  [25],  The  underlying  concept  in  SmaJ.e's 
algorithm  is  the  solution  of  an  ordinary  differential  equation  of  the  form 


F»(x(0))x(e)  = -A(9)  F(x{0)),  x(0)  = 


where  F;  is  continuously  differentiable,  X(0)  is  a real  valued 

function  of  0 such  that  sgn  X = sgn  det  F’(x(0)),  x^  is  specified  in 
R°,  i(0)  denotes  the  derivative  ^ , and  F'(x)  is  the  matrix  of  partial 
derivatives 
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Under  suitable  hypotheses  on  F,  the  solution  of  the  differential  equation 
will  be  a trajectory  x(0)  which  tends  to  a vector  (x*.  x*.  x») 

satisfying 


F(x*)  = 0 . 


(3) 


As  examples,  Smale  shows  that  the  method  relates  to  the  Arrow-BLock-Hurwicz 
dynamics  of  price  adjustment  and  Scarf's  algorithm  for  finding  economic 
equilibria. 

The  computation  of  the  solution  to  (l)  is  normally  achieved  by 
discretization  of  the  differential  equation  and  then  solving  the  discretized 
form  by,  for  example,  the  method  of  continuation  (see  Kello^  li,  and  Yorke 

[14]  or  Davidenko  [2],  and  Meyer  [20]  for  discussions  related  to  this  approach). 
This  implementation  could  lead  to  problems  in  convergence  due  to,  say,  cycling 
or  singularities  of  F'(x). 

In  [lO],  we  proposed  a simplicial  pivot  algorithm  that  in  a precise  limiting 
sense  follows  points  x which  satisfy  the  equation 

F(x)  = tF(x^)  for  some  t £ [O,  l] 


(4) 
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starting  from  a prescribed  yP  (with  t — 1 and  F'(x  ) nonsingular). 

This  method  is  an  improvement  of  a method  proposed  in  [8].  We  shall  show 
in  a later  section  this  simplicial  pivot  algorithm  is  closely  related  to 
Newton's  method  and,  in  fact,  may  be  viewed  as  a globalized  version  of  it. 

We  shall  show  that  the  limiting  sequence  of  points  generated  by  this  method 
will  contain  a solution  to  the  ordinary  differential  equation  (l)  and,  hence, 
the  method,  in  part,  may  be  viewed  as  a constructive  procedure  for  finding 
the  solution  to  Smale's  differential  equation. 

The  present  paper  serves  three  objectives.  The  first  is  to  exposit 
relations  between  the  simplicial  pivot  method  in  [lO],  Merrill's  algorithm, 

Newton's  method,  and  the  method  of  continuation.  The  second  objective  is  to 
exposit  the  relationship  between  the  simplicial  pivot  method  of  [lO]  and  the 
global  Newton' method  of  Smale.  In  conjunction  with  this  exposition  an  example 
is  presented  which  illustrates  the  power  of  starting  at  "infinity."  And  the 
last  objective  is  to  show  some  computational  results. 

We  are  indebted  to  Herbert  Scarf  for  referring  us  to  the  paper  [25]. 

2.  On  the  Instability  of  Merrill's  Algorithm 
Under  a Transformation 

We  here  consider  an  algorithm  for  solving  F(x)  =0  which  is  based 
upon  the  concept  of  complementary  pivoting  on  a labeled  triangulation.  The 
seminal  work  on  complementary  pivoting  and  simplicial  approximation  is  due  to 
Lemke  [17],  Lemke  and  Howson  [l8],  and  Scarf  [23],  [2U].  For  extensions  see 
the  papers  [3],  [»•],  [5],  [6],  [?],  [9],  [12],  [13],  [lU],  [l6],  [19],  and  the 
numerous  additional  references  provided  therein.  We  shall  specifically  consider 
an  algorithm  due  to  Merrill  [l9].  It  was  shown  in  [l]  that  this  algorithm  generates 
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in  a limiting  sense,  the  solution  to  t F(x)  + (l  - t)(x  - x^)  = 0, 
t 6 [0,  1],  X g r”.  Equivalently,  letting  X = - -J'  this  equation 

Xt 

is  written  as 

F(x)  = X(x  - x*^),  X < 0,  X e . (5) 

0 n 

The  algorithm  is  initiated  at  a point  x e R (when  t = 0,  or  X = -“)  • 
Here  F:  -►  R^  is  continuous,  and  X is  a real  variable. 

Merrill's  algorithm  may  be  used  for  finding  zeroes  of  F,  i.e.,  an 
X € eP  satisfying 

F(x)  = 0 . (6) 


Indeed,  it  is  remarkable  that  the  Leray-Schauder  theorem  below,  which  is 
ordinarily  proved  by  arguments  based  on  degree  [22],  can  be  shown  constimctiveiy 
by  letting  F(x)  = x - G(x)  and  x*^  = 0 in  Merrill's  algorithm  (see  [1]): 

Leray-Schauder  Theorem:  Let  C be  an  open  bounded  set  in 

containing  the  origin  and  G:  C -»  R°  a continuous  mapping. 

If  G(x)  Xx  whenever  X > 1 and  x e 9C,  then  G has  a 
fixed  point. 

The  Brouwer  Fixed  Point  Theorem,  which  states  that  any  continuous 
function  from  a compact  convex  set  to  itself  has  a fixed  point,  may  be  easily 
seen  to  follow  from  the  theorem  above. 

Although  the  Merrill  algorithm  is  useful  in  proving  classical  and 
central  theorems  in  fixed  point  theory  by  this  new  constructive  approach, 
there  are  serious  difficulties  in  implementing  the  approach  for  finding 
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zeroes  of  F.  (For  example,  the  algorithm  need  not  converge  even  for  linear 
systems.)  An  analysis  of  system  (5)  shows  that  the  method  is  unstable 

due  to  its  dependence  on  the  indexing  of  the  components  of  F and’  the  signs 
of  the  F^.  For  example,  consider  the  one -dimensional  problem 

F(x)  = = 0 (7) 

where  w > 0 is  a given  constant-  The  zeros  of  F are  x = ±w.  Merrill's 
algorithm,  when  started  at  to  the  right  of  -w,  will  move  towards  w, 

but  when  started  to  the  left  of  -w,  will  move  towards  negative  infinity. 

Replace  F(x)  = 0 with  -F(x)  = 0 and  the  situation  is  reversed:  W'th  yP  < w, 

Merrill's  algorithm  will  move  towards  the  solution  x = -w,  but  with  x*^  > w, 
Merrill's  algorithm  diverges  to  infinity.  Hence,  convergence  is  affected  when 
both  sides  of  the  equation  are  miiltiplied  by  -1,  and  the  particular  root  obtained 
is  also  affected. 

A way  out  of  this  difficulty  is  to  select  x^  such  that  det  F'  (x^)  0 

and  then  to  apply  Merrill's  algorithm  to  the  function 

G(x)  = F'  (x*^)  ^ F(x)  (8) 

as  proposed  by  Fisher,  Gould  and  Tolle  [7 ] . This  Modified  Merrill  algorithm 
finds  solutions,  in  a limiting  sense,  to 

t F(x)  + (1  - t)F'(x^)(x  - x^)  = 0,  t € (0,  1],  X € 

or,  alternatively,  \ (9) 

F(x)  =s  \ F'(x^)(x  - X®),  X < 0,  X € r” 

I 

!Hiis  algorithm  therefore  is  vinaffected  by  a relabeling  or  change  of  signs 
of  the  components  of  F.  Using  (9)  above,  it  is  easy  to  see  that  for  linear 
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systems,  F(x)  = Ax  + b,  A nonsingular,  the  solution  to  (9)  will  be  the 
line  segment  joining  x^  to  the  solution  -A  ^ b,  for  any  x^.  For  the 
problem  (7),  the  Modified  Merrill  algorithm,  when  applied  to  F(x)  = 0 or 
-F(x)  = 0,  yields  the  same  solution,  namely  the  solution  for 

(x^  - w^)  = 2Xx®(x  - x*^),  X < 0 

so  that  for  x^  < 0,  the  method  moves  towards  x = -w  and  for  x*^  > 0,  the 
method  moves  towards  x = w.  Observe  that  the  method  is  still  dependent  on 
the  sign  of  x^.  In  the  simple  case  where  x^  = 0 and  F'(x^)  = I,  for  any 
X < 0,  a point  x such  that  x^  = 0 satisfies  (9)  if  and  only  if  F^(x)  = 0. 
That  is,  the  algorithm  can  move  from  an  orthant  where  x > 0 to  an  orthant  where 
< 0 only  in  a neighborhood  of  points  x for  which  F^(x)  = 0 crosses  the 
plane  x^  = 0.  Moreover,  for  any  x satisfying  (9),  in  the  simple  case  with 
= 0 and  F'(x^)  = I,  F^(x)  must  always  have  a sign  opposite  x^,  whenever 
Fj^(x)  and  x^  are  nonzero. 

It  has  been  shown  that  if  F'(x  ) is  nonsingular,  then  the  Modified 
Merrill  method  is  locally  convergent.  The  underlying  reason  for  the  local 
convergence  behavior  of  the  modified  scheme  is  related  to  the  fact  that  at 
the  initial  txaint  x^  the  algorithm  moves  in  the  Newton  direction. 

To  verify  this  initial  direction,  let  us  suppose  F is  continuously 
differentiable  in  an  oi>en  set  containing  x^,  with  F'(x^)  nonsingular. 

Then  note  that  (9)  can  be  written  as 

H(x,  t)  = t F(x)  + (1  - t)F'(x^)(x  - x*^)  = 0 

where  the  function  H is  continuously  differentiable  on  an  open  set  in 
if-*'-  containing  the  point  x = x^,  t = 0.  Now  we  note  that  the  derivative 
of  H with  respect  to  x is 
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t F'(x)  + (1  - t)F'  (x*^) 

which,  when  evaluated  at  x^,  equals  F'(x^)  (for  any  t).  Then,  by  the 
assumption  that  F'(x^)  is  nonsingular,  the  Implicit  Function  Theorem 
guarantees  the  existence  of  an  open  set  A c containing  t = 0 and  an 
open  set  B c r"^  containing  x^  such  that  for  each  t e A there  is  a unique 
differentiable  function  x(t)  with  values  in  B such  that  H(x(t),  t)  = 0.  Thus 

t F(x(t))  + (1  - t)F’ (x^) (x(t)  - x°)  = 0,  V t € A . (lO) 

Now  differentiating  with  respect  to  t,  and  using  (9), 

[t  F'(x(t))  + (1  - t)F'(x°)]x(t)  = -F(x(t))  + F’(x°)(x(t)  - x°)  = F(x(t))Ul) 

Now  set  X = x^  and  t = 0 to  obtain  (noting  x(o)  = x^) 

F'(x^)x(0)  = -F(x(0)),  or  x(0)  = -(F'(x°))  ^F(x(0))  . 

Since  the  algorithm  is  initiated  with  t increasing  from  0,  x(0)  is  the 

initial  tangent  direction,  and  it  is  seen  to  be  identical  with  the  Newton 
direction.  However,  for  values  of  t 0 the  tangent  directions  as  given  by 
(11)  are  not  Newtonian.  Without  the  nonsingularity  assumption  on  F'(x^)  the 
directions  associated  with  the  original  Merrill  algorithm  applied  to  F ( as 
opposed  to  G defined  by  (8))  are  easily  shown  to  satisfy 

[t  F'(x(t))  + (1  - t)l]x(t)  = ^-^F(x(t))  , 

and  these  directions  are  not  Newtonian,  even  when  t = 0.  Thus,  the 
modification  of  Merrill's  algorithm  at  least  initially  forces  the  path  to  take 
a Newton  direction. 
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In  the  next  section  an  algorithm  is  demonstrated  with  the  property 
that  it  moves  in  either  the  Newton  direction  or  the  negative  of  the  Newton 
direction  at  each  point.  The  attractiveness  of  this  property  will  become 
clear  in  the  sequel. 

3.  On  a Constructive  Global 
Newton  Procedure 

Let  us  now  turn  to  the  simplicial  pivoting  method  proposed  in  [10] 
which  employs  a scalar  labeling  on  a triangTolation  of  to  follow  points 

X which  in  a limiting  sense  satisfy 

F(x)  = t F(x^)  for  some  t g [O,  l]  (12) 

starting  from  a prescribed  x^  € R^  with  t * 1,  where  F:  P.”  -►  r’’  is 

continuous  and  F'(x*^)  exists  and  is  nonsingtilar . The  motivation  for  solving 
(12)  can  be  explained  briefly  as  follows.  Consider  the  classical  problem  of 
how,  given  an  initial  x*^  € one  might  find  a solution  x*-  to  F(x)  = 0. 

Let  us  assume  for  simplicity  (just  for  the  moment)  that  the  preimage  of  the 
line  segment  [0,  F(x^)j  is  a path  x(t),  0 ^ t _<  1,  such  as  when  F is 

a homeoraorphism  on  F ^[O,  F(x^)].  If  in  addition  F'(x(t))  exists  and  is 
nonsingular  for  t 6 [0,  1],  then  this  preimage  is  a differentiable  path 
x(t),  0 ^ t ^ 1.  (See  Meyer  [20]  for  a more  detailed  discussion).  Thus 

F(x(t))  = t F(x®)  , 0 < t < 1 , 

\rtaere  x(t)  is  differentiable.  We  thus  obtain 

F'(x(t))x(t)  = F(x°)  , x(l)  = x° 
or 
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x(t)  = F'(x(t))  ^ F(x^) 

= F*(x(t))-"  for  t >0  . 

This  indicates  that  at  axiy  point  x on  the  path,  at  which  t is  decreasing. 
the  algorithm  moves  in  the  Newton  direction.  In  particular,  since  the 
algorithm  is  initiated  at  t = 1,  with  t decreasing,  the  initial 
direction  of  the  algorithm  is  -(F'(x*^))  ^ F(x*^).  This  scheme  is  illustrated 
in  Figure  1,  where 


x^  = x*^  - x(l)  = x^  - F'(x*^)  ^ F(x*^)  . (13) 


In  our  context,  Newton's  method  may  therefore  be  viewed  as  an  attempt  to 
follow  the  preimage  of  [0,  F(x^)]  by  taking  the  next  point  x^  to  be  in 
the  direction  of  the  tangent  vector  to  the  preiraage  curve  at  t = 1. 

As  is  well  known,  a sequence  of  Newton  iterations  can  be  guaranteed 

0 "it 

to  converge  only  when  x is  close  to  x and  F'(x  ) is  nonsingular. 
Continuation  methods  may  be  used  to  widen  the  domadn  of  convergence  of 
Newton’s  method.  This  approach  has  been  known  for  the  last  century  amd 
has  been  rediscovered  many  times  since.  Essentially,  the  interval  [0,  1] 
is  partitioned  into  subintervals 


k k + 1 
T " T 


k = 0,  1,  . . . T - 1 


(li^) 


so  that  for  large  positive  integers  T,  the  paths  x(t)  on  each  subinterval 
are  "small  enough"  so  that  convergence  is  obtained  when  Newton’s  method  is 
applied  on  each  subinterval. 
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One  known  variation  [22]  of  the  method  works  as  follows.  Consider 
^ T - 1 T - 2 

the  values  t = 1,  — j j 0 and  the  associated  path  points 
x(l)  = x^,  xC*^  ~ = x^,  x('^  ~ = x^,  x(0)  = x^  = X*.  If  each  x^ 

is  on  the  path  (12),  theu 


F(x^)  = F(x^),  F(x^)  = F(x^),  and  F(x^)  = F(x°)  . 


Step  1.  Start  at  x . Take  the  first  Newton  step  for  solving 


F(x)  - F(x  ) = 0 . 


This  produces  a point 

= x°  - F’(x°)'^  (F(x°)  - F(x^))  = x°  - F'(x°)"^  ^(x°)  + 

Step  2.  Start  at  y^.  Take  the  first  Newton  step  for  solving 

F(x)  - F(x^)  = 0 . 


This  produces  a point 

y^  = y*-  - F’(y'-)  ^ (F(y^)  - F(x^))  = y^  - F'(y^)"^  ^F(y^)  + ^~^--'^F(x°)^ 

T-1 

Step  T.  Start  at  y . Take  the  first  Newton  step  for  solving 
F(x)  - F(x^)  =0,  or,  since  F(x^)  = 0,  F(x)  = 0 . 

This  produces  a point 

T T-1  , T-lv-1  T-1. 

y = y - F' (y  ) F(y  ) . 


Thus  we  have  the  following  sequence  of  iterates,  where  y*^  = x*^, 

- F'(y^r^  ^FCy'^)  + F(x°)j,  k = 0,  1,  ...,  T - 1 . (15) 

As  Figure  2 indicates,  at  any  "initial  point"  y the  Newton  step  is  the  initial 
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■ direction  of  the  path 

F(x(t))  = (1  - t)F(x^'^^)  + t F(y^),  0 < t < 1 . 

The  objection  to  this  approach  is  that  in  order  to  assure  convergence 
to  X*,  starting  from  x^,  nonsingularity  of  the  Jacobian  of  F is  required 
at  every  point  of  the  path. 

The  purpose  of  [10]  was  to  show  that  the  preimage  of  [0,  F(x*^)  ] can 
be  follOTed  under  less  stringent  conditions  on  F.  It  can  be  shown  that : 

Theorem  1.  If  F:  -*  is  continuously  differentiable  at  x*^, 

F' (x^)  is  nonsingular,  F~^[0,  F(x*^)]  is  compact,  and  F(x)  = F(x^) 
iff  X = x*^,  then  there  is  an  x such  that  F(x)  = 0. 

This  theorem  is  constructively  shown  in  [8].  The  difficult  part  in  the  proof 

of  the  theorem  above  is  to  show  that  the  scheme  has  a unique  starting  simplex 

at  t = 1.  But  this  is  assured  in  [12,  Proposition  3-5]  which  shows  that  for 

triangulations  with  mesh  size  small  enough,  the  continuous  simplicial  mapping 

used  to  approximate  F will  also  have  a lanique  zero  in  a neighborhood  of  x^, 

under  certain  "regularity"  assimptions  on  F and  the  triangulation. 

The  scheme  was  improved  in  [10]  by  presenting  a scalar  labeling 

version  of  the  method.  The  computational  burden  was  improved  to  that  of 

2 

order  n per  pivot  as  opposed  to  an  order  (n  + 1)  per  pivot  (see  Table  1) . 
A similar  comparison  can  be  made  between  the  Merrill  algorithm  [19  ] and  the 
scalar  version  introduced  in  [9l- 

Observe  that  the  path  generated  by  (12)  and  described  in  [lO]  is  a 
substantial  improvement  to  Newton's  method  in  terns  of  global  convergence. 
Newton's  method  could  cycle,  or  it  could  "blowup"  when  the  Jacobian  at  some 
point  is  singular.  Scheme  [10]  on  the  other  hand  will  succeed  in  generating 
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an  approximate  solution  even  when  these  cases  occur,  so  long  as  the  conditions 
of  Theorem  1 above  are  satisfied.  In  fact,  the  condition  F(x)  = F(x^)  iff 
X = x^  is  also  needlessly  strong,  and  this  assumption  will  be  dispensed  with 
in  Theorems  T and  8 of  the  next  section. 

Scheme  [10]  has  several  other  properties  of  interest.  Th.s  method  is 
Jacobian  invariant  and  norm-reducing.  The  former  property  is  desirable  since 
we  would  have  convergence  unaffected  by,  say,  a relabeling  of  connxDnents  of  F. 
By  the  latter  property  we  mean  that 

||F(x^)|l  < |1f(x‘'‘^)|1  (16) 


i. 


I 


L 


where  x is  the  approximate  solution  at  the  k — iteration.  This  norm- 
reducing  property  seems  to  be  a property  not  shared  by  other  simplicial 
pivot  procedures. 

There  is  another  possibility  that  is  furnished  by  scheme  [10].  If 
k 

X is  a point  on  the  (n  - 1) -simplex  generated  after  k pivots,  then 

the  vector  x - x , \diere  k is  a "few"  iterates  after  k,  is  approximately 

the  Newton  direction  (or  the  negative  Newton  direction) . In  fact 


P(x‘")  = F(x°) 

F(x‘")  S tj^  F(x°) 

for  some  0 < t < 1 so  that  if  it  can  be  shown  that 

. k ■ "k 

- -tk^" 

X = 7 r 


(17) 


(18) 


will  be  an  approximate  Newton  step.  Thus,  without  much  additional  work,  we 
can  consider  taking  the  Newton  step  if  | |f(x)| | is  significantly  smaller 
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than  either  of  [ | F(x  ) | | or  | | F(x  ) | | , and  restarting  the  scheme  at  x, 
where  x is  defined  by  (l8) . 

4 . Relation  to  Smale's  Global  Newton  Method 

In  this  section  we  relate  the  set  of  solutions  to 

F(x)  = tF(x°),  X e r”  , t e R (19) 

to  the  set  of  solutions  to 

F’(x(e))x(0)  = - X(0)  F(x(9)),  x(0)  = x°  where  sgn  X = sgn  det  F'(x(6))i20) 

Equation  (20)  is  the  differential  equation  introduced  by  Smade  in  [25  ].  Here 
X is  some  real  function  of  9,  such  as,  for  example,  X - det  F'(x(0)). 

Let  us  make  a quick  observation  on  the  solution  to  (20)  . Assume  the 
solution  is  a one-dimensional  curve  x(9)  , 0 < 0 < 1 where  x(0  ) = x^  is  some 
given  starting  point,  and  x(l)  = x*  is  a zero  of  F.  Then  starting  from  x*^ 
the  curve  x(0)  is  such  that  at  any  regular  point  x in  the  path  (i.e., 

det  F'(x)  ^ 0)  the  vector  x(0),  the  tangent  to  the  curve,  is  either  the 
Newton  direction  (if  X is  positive)  or  the  negative  Newton  direction  (if 
X is  negative).  The  change  of  direction  occurs  at  points  x(0)  where  X(6) 
becomes  zero  and  hence  where  the  Jacobian  becomes  singular.  Indeed,  one  shoiild 
note  that  the  solution  trajectory  for  equation  (20)  is  a globalization  of 
Newton’s  method  in  the  sense  that  the  curve  is  "continued"  when  singularities 
of  the  Jacobian  are  encountered. 

Now  consider  equation  (l9).  This  equation  generalizes  the  siraplicial 

pivot  algorithms  of  [8,  lO)  in  the  sense  that  the  parameter  t is  no  longer 

2 

restricted  to  the  unit  interval.  We  will  show  that  for  C functions  F with 
F’(x*^)  nonsingular  a piece  of  the  solution  to  (19)  is  the  solution  to  (20).  We 
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dropped  the  condition  0 £ ^ £ 1,  which  was  assumed  in  [lO],  because 

this  enables  us  to  better  deal  with  the  case  where  the  preimage  of  F(x^) 

is  not  unique,  and,  moreover,  under  appropriate  conditions  it  allows  us  to 

continue  the  paths  so  as  possibly  to  obtain  all  zeroes  of  F. 

2 

Let  us  consider  a C (twice  continuously  differentiable)  function 
H:  -*  r”.  Given  y e R°,  let 

H"^(y)  = {x  e r^'^^IhCx)  = y}  (21) 

and 

C = {x  e R°'^^|rank  H'(x)  < n}  (22) 

where  H'  is  the  Jacobian  matrix 

The  set  C is  said  to  be  the  set  of  crticial  points  of  H,  and  H(C)  the 
set  of  critical  values.  ~ H(C)  is  the  set  of  regular  values . Sard's 

Theorem  [26  ] states  that : 

Theorem  2.  Let  H:  -►  r"  be  a map.  Then  H(C)  has 

measijre  zero. 

Thus,  as  a corollary,  the  set  of  regular  values  is  dense  in  R*^.  Let  us 
henceforth  throughout  this  paper  assume  that  0 is  a regular  value  of  H. 

The  following  lemma  will  be  used  [2I 1 : 

Lemma  3.  Let  H:  R*'^^  -►  r”  be  a map  and  let  0 be  a regular 

value  of  H.  Then  H~^(0)  is  a one-dimensional  manifold. 


3H. 

1 


dx 


of  H with  respect  to  x € ^ 


n+1 


1 
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Now,  recall  that  einy  connected  one-dimensional  manifold  is 

diffeomorphic  to  a circle  or  an  interval,  (open,  closed,  or  half -open) . Thus,  each 
Cconnected) component  of  H ^(O)  can  be  described  by  a curve  x(9)  which  is  diffeo- 
morphic to  a circle  or  an  interval.  Furthermore,  for  any  x(0)  e H~^(0),  we 
have 

rank  H'(x(0))  = n (23) 

and  x(§)  is  a unique  nonzero  vector.  Consequently,  we  can  differentiate 
H(x(0))  = 0 with  respect  to  0 to  obtain 

H'(x(9))  x(e)  =0  . (24) 

For  a particular  6,  x(0)  is  a vector  tangent  to  the  curve  at  0 = 0 and  spans 

the  kernel  of  H'(x(9)). 

For  any  i = 1,  2,  ...,  n + 1,  let  x^(0)  and  H^(x(6))  denote  the 

component  of  x(0)  and  the  i^^  column  of  H'(x(9)),  respectively,  and 

let  x^(9),  H^(x(9))  be  the  remaining  components  of  x(9)  and  col\imns  of 

2 

H'(x(9)),  respectively.  The  following  theorem  for  C maps  is  related  to  and 

motivated  by  a theorem  of  Eaves  and  Scarf  for  piecewise  linear  maps  [4].  The 
proof  of  the  theorem  appears  in  [ll]. 
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Theorem  ^ . Let  H:  "•  be  a map  and  0 a regular  value 

of  H.  Then  for  any  component  x(0)  of  H"^(0)  we  have  for  all 
i = 1,  2,  . . n + 1: 

sgn  x^(9)  = sgn  det  H^(x(Q))  all  0 (25) 

or 

sgn  x^(0)  = -sgn  det  H^(x(©) ) all  0 (26) 

(where  sgn  0 = 0) . . 

Note  that  the  theorem  holds  if  H is  restricted  to  say,  R^  x [o,  l].  In 
most  applications  to  r"^  x [o,  l]  a further  restriction  would  be  required 
on  the  boundaries  R*^  x o {O}  and  r’^  x {i} — naimely,  nonsingularity  of  the 
n X n submatrix  H^(x>  't)  at  points  (x,  t)  in  the  boundary  for  which 
H(x,  t)  = 0.  The  condition  assiires  that  all  loops  which  occur  are  contained 
in  r”  X (0,  1),  An  interesting  corollary  to  Theorem  4 is  the  following 
monotonicity  theorem. 

Corollary  5.  Let  H:  ->■  R^  be  a map  and  0 a regular 

value  of  H.  Suppose  for  some  i,  H^(x)  is  nonsingular  for  all 
x in  a particular  component  x(0)  of  H~^(0).  Then,  on  that  component 
of  H ^(O),  Xj^(9)  is  either  monotone  increasing  or  monotone  de- 

creasing as  a function  of  9. 

Observe  that  under  the  assumptions  of  Corollary  5 the  distinguished  component 
of  H~^(0)  cannot  be  diffeomorphic  to  a circle. 
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Let  us  now  turn  to  equation  (l9),  where  F is  assumed  to  be  a 
2 0 

C map  with  F'(x  ) nonsingular.  We  associate  with  (19)  the  homotopy 
H:  defined  by 

H(x,  t)  = F(x)  - tF(x°)  , X e r“  , t e R (27) 

2 *2.  X 

which  is  of  course  aLLso  C . By  Lemma  3,  H (O)  is  a C one-dimensional 
manifold,  so  that  the  component  of  H ^(0)  containing  the  initial  point 
(x^,  1)  may  be  described  by  a curve 

(x(e),  t(o))  , (x(o),  t(o))  = (x^,  1)  . 

Let  us  visualize  the  complementary  pivoting  scheme  [lO]  to  be  tracking 
this  curve  in  terms  of  increasing  0,  Observe  that  equation  (2U)  in  this  special 
case  reduces  to: 


F'(x(0))x(e)  = t(©)F(x°)  = F(x(9))  (28) 

if  t(©)  ^ 0.  Thus  if  t(0)  ^ 0,  it  follows  from  Theorem  h that 
det  F'(x(9))  # 0,  so  that 

x(©)  = F'(x(©))"^  F(x(©))  . (29) 

An  examination  of  (29)  yields  the  following  conclusions: 

i)  If  t(©)t(©)  > 0,  then  scheme  [10]  is  moving  in  the  negative 
NPwt.on  direction. 

ii)  If  t(©)t(©)  < 0,  then  scheme  [10]  is  moving  in  the  Newton  direction. 
Figure  3 below  gives  a graphical  picture  of  this  result. 
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As  an  illustration,  let  (x(9),  t(9))  be  the  curve  shown  in 
Fig\u-e  3 where  9 is  increasing  in  the  direction  of  the  arrow,  and 
(x(0),  t(0))  = (x^,  1).  Since  t(0)t(0)  < 0,  we  are  initially  moving 
along  the  Newton  direction.  We  continue  movement  along  the  Newton  direction 
until  we  get  to  a point  a,  where  t(9)  = 0.  From  point  a to  point  b, 
since  t(9)  is  now  positive,  and  t(9)  is  still  positive,  we  are  moving 
along  the  negative  Newton  direction.  From  b to  c,  with  t(9)  negative, 
we  move  along  the  Newton  direction  again.  At  point  c,  where  we  have  a 
zero  of  F,  we  again  reverse  direction  since  t(9)  changes  sign.  We  again 
reverse  direction  at  points  d and  e. 

Thus,  the  direction  reverses  if  either  t(©)  or  t(9)  changes  in 
sign,  but  not  both.  The  value  t(©)  changes  sign  at  a zero  of  F(x) . It  follows 
from  Theorem  U that  t(9)  changes  sign  only  when  det  F'(x(9))  = 0, 

We  now  use  Theorem  U to  relate  completely  equations  (19)  and  (20) . In 
this  specie^,  case,  the  theorem  reduces  to: 
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Theorem  6.  sgn  t(©)  = sgn  det  F'(x(Q))  all  0 
or  sgn  t(9)  = -sgn  det  F'(x(©))  all  © . 

Observe  that  the  differential  equation  (28)  corresponding  to  (l9)  is 

equation  (20)  if  sgn  = -sgn  det  F'(x(©)).  To  see  that  this  equality  in 

fact  holds,- recall  that  t(0)  =1,  and  suppose  ve  initially  move  in  a direction 

with  t decreasing  from  1 if  det  F'(x^)  >0.  Otherwise,  if  det  F'jx)  < 0, 
. . - - ^ V t(0)  _ 


ye  let  t initially  increase  from  1*  (See  Figure  3*)  Then  sgn 
-sgn  det  F'(x°).  If  we  do  indeed  choose  this  initial  direction  to  move, 

t(©) 

Theorem  9 tells  us  that  sgn  t(©)  = “sgn  det  F'(x(©))  all  ©.  Thus  sgn 
-sgn  det  F'(x(©))  for  all  © where  t(©)  >0,  and  for  such  values  of  0 our 
scheme,  equation  (19),  is  in  fact  a solution  to  Smale's  equation  (20).  This  actually 
pjToves  that  a piece  of  the  solution  to  (19)  is  the  solution  to  (20),  for  Smale  s 
algorithm  terminates  the  moment  the  first  zero  is  encountered  (t(0)  = 0).  In 

Figure  3,  the  path  from  (x°,  1)  to  point  c is  the  SmeLIe  path. 


The  reader  should  now  relate  the  behavior  of  the  path  to  the  Jacobian 

F'(x(©)).  If  det  F'(x°)  > 0,  then  we  move  in  the  Newton  direction  for  all 

© where  t(©)  det  F'(x(©))  > 0 and  in  the  negative  Newton  direction  for 

t(0)  det  F'(x(©))  <0.  The  det  F'(x(©))  changes  sign  as  t(©)  changes.  Thus 

in  Figure  3,  det  F'(x(©))  > 0 for  points  from  (x®,  1)  to  a,  and  from 

b to  d , and  det  P’(x(©))  < 0 for  points  a to  b,  and  from  d to  e.  Finally, 

det  F'(x(0))  =0  at  points  a,  b and  d. 

In  his  paper  l25  J Smale  presented  a new  and  interesting  convergence  result.  He 

considers  an  open  bounded  set  C of  r”  together  with  its  boundary  dC . 

Assume  that  C and  30  are  connected  secs,  and  tha'  3C  is  smooth.  Suppose 

2 

P;  C “•  is  a given  ^ function,  and  assume  that  F satisfies  a boundary 
condition,  e.g.: 
f 


2h 

Boundary  Condition.  For  each  x e 9C,  det  F'(x)  >0  and  F'(x)~^F(x) 
intersects  9C  transversally  at  x (i.e.,  F'(x)  ^F(x)  is  not  tangent 

to  BC  at  x)  . 

This  implies  that  on  BC  the  Newton  direction  is  "everywhere  pointing 
into  C"  or  "everywhere  pointing  out  of  C."  Smeile  then  gives  the  following  theorem 
a different  proof  of  which  is  here  included  for  the  sake  of  completeness. 

• n 2 

Theorem  7 . Let  F:  C "*  R be  a C function  where  F and  the 
open  bounded  C ^ satisfy  the  boundary  condition,  where  C and  3C 

connected  with  3C  smooth.  Then  for  x*^  e BC  such  that  0 is  a 
regular  value  of  H defined  by  (27),  the (connected)  component  of 
H ^(O)  containing  (x*^,  1)  will  contain  a zero  of  F. 

Proof;  Since  F:  C R^  we  have  H:  C x R where  H(x,  t)  = 

F(x)  - tF(x^) . 

Let 

H'^(O)  = {(x,  t)  e c X R:  H(x,  t)  = o]  . 

Since  0 is  a regular  value  of  H (i.e.,  [H  , H ] has  rank  n V (x,  t)  6 
H ^(O)),  and  since  H ^(O)  is  closed,  the  component  of  H~^(0)  containing 
(x^,  1)  may  be  described  by  a curve  (x(S),  t(©)),  where  (x(0),  t(0))  = 

(x^,  1)  and  0 e [0,  1].  This  component  cannot  be  a loop.  That  is  (x(l), 

■t(l) ) / (x*^,  1).  This  conclusion  follows  from  the  fact  that  the  path  (x(0), 
t(0))  is  , BC  is  smooth,  and  hence  if  the  path  were  a loop  the  trans- 

versality  boundary  condition  would  be  violated.  But  if  the  path  is  not  a loop 
then  the  endpoint  (x(l),  t(l))  must  lie  in  B(C  x R)  . Otherwise,  since  0 is 


are 


25 


a regular  value  of  H,  rank  [H  (x(l),  t(l)),  H (x(l),  t(l))]  = n,  and 
by  the  Implicit  Fiuiction  Theorem  x(l),  t(l)  must  be  interior  to  H ^(O). 

Thus,  the  point  x(l)  must  lie  in  3C.  We  now  show  that  this  implies 

t(l)  < 0 and,  since  t(0)  = 1,  there  is  a 6*  g (0,  l)  for  which  t(9*)  = 0 

and  hence  F(x(0*))  = t{0*)  F(x^)  = 0. 

To  see  that  t(l)  < 0,  recall  the  boundary  assumption  det  F'(x)  > 

0 V X f 3C.  By  Theorem  6,  and  the  convention  that  t is  initially  negative 

(i.e.,  t(0)  < 0),  we  have  t(0)  < 0 V 0 such  that  x(0)  € 3C.  In  particular, 

• • 

t(l)  < 0.  The  convention  that  t(0)  <0  is  tantamount  to  assuming  that  the 
Newton  direction  must  point  inward  at  each  x ^ 3C,  including,  in  particular, 
the  point  x(l).  This  means  that  at  0 = 1 the  tangent  x(0)  is  in  the 
negative  Newton  direction.  This  implies,  by  the  remarks  following  (29),  that: 

t(l)t(l)  > 0 . 

Since  t(l)  <0  it  must  also  be  true  that  t(l)  <0.  K 

Let  us  drop  the  condition  that  det  F'  (x)  > 0 for  all  x e bc,  and 
then  add  the  regularity  condition  that: 

F'(x)  is  nonsingular  for  each  x e F ^(O)  . (30) 

Note  that  (30)  guarantees  the  finiteness  of  zeroes  of  F in  C.  In 
fact  we  can  show  the  parity  of  the  ntunber  of  zeroes  of  F in  the  component 
of  H”^(0)  containing  (x^,  1) . 


I 

1 
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Theorem  9,.  Let  F:  C “*  r”  be  smooth  where  C ^ r”  is  open  and 
bounded,  C and  9C  are  connected.  Then  for  e dC  such  that  F(x°) 
0 and  0 is  a regular  value  of  H defined  by  (27) , the  connected 
component  (x(9) , t(0)),  0<e<l  of  H"^(0)  with  (x(0),  t(0))  * 
(x^,  1)  will  contain  an  odd  (even)  number  of  zeroes  of  F if  and 
only  if  t(l)  < 0 (>  0) . 

Proof:  If  the  component  of  H~^(0)  containing  (x*^,  1)  is  the 

singleton  (x^,  1),  the  theorem  holds  trivially.  Otherwise,  as  in 
the  case  of  the  last  theorem,  the  component  is  a curve  (x(0),  t(©)) 
with  (x(0),  t(0))  = (x°,  1),  x(0)  e c,  0 < 0 < 1,  and  x(l)  e ac, 

where  here  we  allow  the  possibility  of  a loop,  i.e.,  (x(0) , t(0) ) = 

(x*^,  1)  = (x(l),  t(l)).  Now  let  0 < © < 1 be  such  that  F(x(0))  = 0 
(so  that  t(©)  = 0).  Since  F'(x(0))  is  nonsingular  by  assumption  (30), 
it  follows  from  Theorem  6 that  t(©)  ^ 0.  Hence  there  is  an  e > 0 
such  that  t(0  - a)  • t(©  + a)  < 0 for  0 < a < e . Thus  t(0)  changes 
sign  on  some  interval  if  and  only  if  a zero  of  F is  encountered  on 
the  interval.  Hence,  if  t(l)  < 0,  since  t(0)  > 0,  the  sign  of 
t(0)  must  have  changed  signs  an  odd  number  of  times  implying  an  odd 
number  of  zeroes  of  F on  the  curve.  Otherwise,  if  t(l)  > 0,  the 
sign  of  t(0)  changed  signs  an  even  (possibly  zero)  number  of  times 


implying  an  even  number  of  zeroes  of 


F on  the  curve. 
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5 • Examples 

Example  1:  Comparison  of  Global  Newton  and  Modified  Merrill 

We  recall  that  scheme  [lO]  requires  o(n)  arithmetic  operations  per 

• . . . . P 

pivot  while  the  Modified  Merrill  vector  labeling  algorithm  requires  o(n  + l) 

operations  per  pivot.  Furthermore,  the  two  schemes  follow  quite  different  paths 
but  initially  start  in  the  same  direction.  An  example  is  shown  in  Figure  4. 

Here  the  Modified  Merrill  diverges  to  infinity  while  scheme  [10]  converges  to 
a solution.  Observe  the  dependence  of  the  former  algorithm  on  the  sign  of  x^. 

In  fact  a hyperplane  x^  = 0 cannot  be  crossed  by  the  limiting  path  of  the 
Modified  Merrill  algorithm  except  at  a point  on  the  hyperplane  where  F^(x)  = 0 
also.  The  scheme  [lO]  requires  t C [0,  1]  so  that  the  solution  is  bounded  by 
the  intersection  of  sets  0 ^ F^(x)  ^ F^(x*^)  and  is  not  dependent  on  the  signs 
of  x^.  Observe  too  that  the  solution  to  the  more  general  equation 

F(x)  = tF(x^),  t real 

is  a loop  that  connects  all  preimages  of  F(0)  and  all  zeros,  so  that,  if 
the  algorithm  is  started  at  the  origin,  it  will  find  all  zeros  of  F, 

Example  2:  Starting  at  Infinity 

Figure  5 raises  the  interesting  possibility  that  for  certain  types 

2 

of  functions  convergence  of  G may  be  assured  by  starting  sufficiently 

far  away  from  a zero.  Concerning  this  figure,  we  make  the  following  observations 

(i)  G fails  when  initiated  from  w (a  loop  is  generated). 

2 0 
(ii)  G succeeds  when  initiated  far  away,  e.g.,  from  x , and 

moreover  all  zeros  are  found. 
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(iii)  The  set  C with  the  ciirly  border  satisfies  the  hypotheses 
of  Smale  (Theorem  7 ).  In  particular  det  F'(x)  > 0 for 
«Q.l  X € 3C  auid  the  Newton  direction  points  into  C at 
each  X € 3C. 

6.  Empirical  Comparisons 

We  have  compared  the  Modified  Merrill  vector  labeling;  method  with 
G , the  scalar  labeling  method  from  [lO],  on  three  nonlinear • problems . All 
tests  were  done  on  an  HP  2000  Time-Sharing  Basic. 

The  first  example  is  to  solve: 

F^(x^,  Xg)  = x|  - x^  - 1,000  = 0 
FgCx^,  Xg)  = x^  - X2  - 1,000  = 0 . 

An  approximate  solution  to  the  problem  is  x^  = Xg  = 10.0333  with  | |f(x)|  * 

1.29  ^ 10”^.  Here  Merrill's  algorithm  (without  modification)  did  not  converge 
using  a mesh  size  of  one  unit  and  starting  point  x = (20,  20)  (after  351 
iterations,  the  method  generated  the  point  x = (-66,  77)  simplicial 

path  is  diverging  to  infinity).  Both  the  Modified  Merrill  aind  scheme  [lO]  traverse 
the  line  segment  Joining  the  starting  point  and  the  solution. 


TABLE  2 (Number  of  Pivots) 
Starting  point  x®  » (20,  20) 


Mesh  size 

.01 

.1 

1 

5 

Modified  Merril.!  | 

699 

69 

13 

Scheme  [ 10 ] | 

3,985 

397 

37 

5 

■*The  Modified  Merrill  was  stopped  after  the 
indicated  number  of  simplicial  pivots  without  reaching 
the  terminal  sizsplex. 
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The  result  of  the  test  is  shown  in  Table  2.  In  every  case, 
scheme  [lO]  took  fewer  pivots  than  the  Modified  Merrill  to  reach  the 
terminal  simplex.  This  is  explained  by  the  fact  that  Modified  Merrill 
operates  in  while  operates  in  r”.  Moreover,  the  M^  path 

(projected  onto  r'^)  will  generally  differ  from  the  path,  as  Example  1 shows. 

This  also  will  cause  the  two  algorithms  to  generate  different  numbers  of 
total  pivots.  Due  to  the  significant  difference  in  the  computational 
burden  of  the  Modified  Merrill  (o(n  + 1)  operations  per  pivot)  over  scheme 
[lO],  (o(n)  operations  per  pivot),  the  total  computational  work  in  the 
latter  scheme,  given  approximately  the  same  number  of  pivots,  is  significantly 
less  than  the  former  scheme. 

It  is  interesting  to  note  that  for  mesh  size  equal  to  e,  scheme  [ 10] 
takes  about  pivots  to  reach  the  ter.minal  simplex,  which  is 

roughly  the  nxnaber  of  pivots  required  to  reac.h  the  terminal  simplex  in  the 
quickest  way  (see  Figure  6 ) . 
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V/e  ran  both  scher.es  on  the  sar.e  problem  but  using  as  initial  point 
X = (2C,  50)  • '^-2  result  cf  the  test  is  exnibinsd  in  Table  3 


TABLE  3 (Number  of  Pivots) 
Starting  point  = (20,  30) 


Mesh  size 

.1 

1 

5 

10 

1 

Modified  Merrill  | 

1,099 

109 

21 

10 

Scheme  QlO]  j 

597 

51 

Q 

y 

3 

The  second  problem  tested  is: 

F^(x^,  Xg)  = 2x^  + sin  x^  - X2  - 101  = 0 
Fg(xi,  X2)  = + sin  Xg  - - 102  = 0 , 

It  was  shown  in  [10  J that  in  a 5~<iinension3l  proble.m  of  this  form 
both  Ne'.fton ' s method  and  the  continuation  method  diverged.  An  approximate 
solution  to  this  problem  is  x = (ICO.S99,  101.157)  with  ||F(x)1I„  = 2.^  x lO"^. 
Again,  as  shown  in  Table  scheme  [10  J is  superior  to  the  Modified  .Merrill  in 
this  example. 


3U 


TABLE  U (Number  of  Pivots) 
Starting  point  = (20,  20) 


Mesh  size 

1 

5 

10 

Modified  Merrill 

569 

115 

59 

Scheme  [10  ] 

323 

51 

33 

The  third  and  last  problem  tested  is: 


F.(x) 


Z X.  - 1,000  = 0 


We  ran  both  schemes  on  the  above  for  n =5,  10  and  x?  = 20  ai  i 
The  resxilts  are  shown  in  Tables  5 and  6. 


TABLE  5 (Number  of  Pivots) 
n = 5 Starting  point  x°  = 20  i = 1,  2,  ...,  5 


Mesh  size  | 

1 

5 

10 

Modified  Merrill  | 

U96 

S6 

U6 

Scheme  [10] 

. 369 

73 

2h 

TABLE  6 (Number  of  Pivots) 
n » 10  Starting  point  x^  = 20  ali  i 


Mesh  size 

5 

10 

Modified  Merrill 

5^1 

266 

Scheme  [10] 

**53 

20^" 
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